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DETAILED ACTION 



Specification 

1 . The title of the invention is not sufficiently descriptive. A new title is required that 
is more clearly indicative of the invention to which the claims are directed. 



Drawings 

2. The drawings are objected to because in Fig. 3 the drawings do not agree with 
the specification. In block S1 1 of Fig. 3, it specifies that if cnt is less than equal to 
threshold k than to proceed to E2, however the specification on pages 10 lines 25-26 
and page 1 1 line 1 specify the opposite. In Fig. 5, "toward" should be changed to - 
forward-, to be consistent with the Specification. 

The objection to the drawings will not be held in abeyance. 

3. Figure 20a should be designated by a legend such as --Prior Art-- because only 
that which is old is illustrated. See MPEP § 608.02(g). Corrected drawings in 
compliance with 37 CFR 1 .121(d) are required in reply to the Office action to avoid 
abandonment of the application. The replacement sheet(s) should be labeled 
"Replacement Sheet" in the page header (as per 37 CFR 1 .121 (d)) so as not to obstruct 
any portion of the drawing figures. If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 
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Claim Objections 

4. Claims 3, 5 and 6 are objected to because of the following informalities: 
In claims 3 and 5, on pages 33 and 34, lines 16 and 21, respectively, the 

applicant recites "the extracted frequency vectors". This would suggest that extracted 
frequency vectors had been mentioned previously, but they have not. Therefore, "the" 
should be removed. 

Claim 5 also states on lines 24 and 25 "a frequency vector calculate for 
calculating frequency of characteristic symbols". This should be changed to -a 
frequency vector calculator for calculating the frequency of characteristic symbols--. 

In claim 6, the applicant recites "extracting and frequency vectors" on line 9. This 
should be changed to -extracting the frequency vectors--. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1-6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Dutta 
et al. (U.S. Pat. 6,453,294) in view of Challapali (U.S. Pat. Pub. 2002/01940006). 

As per claim 1, Dutta teaches a text to speech synthesizer, comprising: 
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a facial character reading assignment unit for assigning sounds to character 
string portions of text analysis results determined to correspond to facial characters 
(converts emoticons to sound, col. 3, lines 65-67); 

a speech synthesizer for outputting synthesized speech based on the analysis 
results of the text analyzer (text-to-audio transcoding, col. 3, lines 60-61), wherein the 
facial character reading assignment unit is constituted by; 

a facial character determining unit for determining whether or not a symbol is a 
symbol constituting a facial character (converts emoticons to sound, therefore 
necessarily detecting them, col. 3, lines 65-67); 

determines if text is a command using an outline symbol table (commands 
placed in brackets, col. 4, lines 16-18); and 

a characteristic extraction unit for extracting characteristic symbols used in facial 
characters from facial character strings determined to be for facial characters and 
assigning facial characters corresponding to characteristic symbols (multiple emoticons 
are recognized that are made up of different characters hence the differences in the 
characters would differentiate the emoticons, col. 3, lines 47-53). 

Dutta does not teach analyzing Japanese text data. 

However, the Examiner takes Official Notice that analyzing Japanese text for 
synthesis is well-known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of invention to modify the system of Dutta to analyze 
Japanese text data because it would allow the system to be used by those who do not 
speak English. 
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Dutta does not teach using an outline symbol table to determine whether a 
symbol is a symbol constituting a facial character. 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the system of Dutta to determine whether a symbol is a symbol 
constituting a facial character using the outline symbols for the commands because this 
would allow the system to quickly and easily differentiate between facial characters and 
text so that these facial characters are not processed as regular text. 

Dutta does not teach a reading selection unit for outputting readings allotted to 
extracted reading numbers. 

However, the Examiner takes Official Notice that storing data with a pointer to 
corresponding data is common in the art. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time of invention to modify the system of Dutta so that 
the reading selection unit to output readings allotted to extracted reading numbers 
because it would save memory. 

Dutta does not teach the readings being allotted to the facial character strings 
according to the number of appearances of characteristic symbols in the facial 
characters. 

Challapali teaches the readings being allotted to the facial character strings 
according to the number of appearances of characteristic symbols in the facial 
characters (emoticons have repeated symbols hence the number of symbols would 
influence the allotment of the reading, Appendix). 
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It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the system of Dutta to have the readings being allotted to the facial 
character strings according to the number of appearances of characteristic symbols in 
the facial characters as taught by Challapali because determining the number of 
symbols in the facial characters would be a quick method to reduce the searching for 
the corresponding reading. 

7. As per claim 2, Dutta teaches the facial character reading assignment unit 
decides upon readings for facial characters using the steps of: 

scanning the text and detecting a left and right outline symbol listed in the outline 
symbol table (detect left and right brackets to determine command, col. 4, lines 16-18); 

extracting symbols exhibiting characteristics of eyes from a character string 
portion (emoticons have symbols that represent the eyes, col. 3, lines 47-53); and 

referring to the characteristic symbol table and the reading table, and deciding 
upon a corresponding facial character reading from readings for characters exhibiting 
eyes (converts emoticons which exhibit eyes to sound, therefore necessarily detecting 
them, col. 3, lines 65-67). 

Dutta does not teach detecting a right outline symbol within a range of a 
prescribed number of characters if a left outline symbol is detected. 

Challapali suggests the facial characters are only a few symbols in length 
(emoticons have a maximum of six characters, Appendix). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of invention to modify the system of Dutta to detect a right outline symbol within a 
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range of a prescribed number of characters if a left outline symbol is detected because 
it would limit the search for the facial characters hence speeding up processing. 
8. As per claim 3, Dutta teaches a text to speech synthesizer, comprising: 

a facial character reading assignment unit for assigning sounds to character 
string portions of text analysis results determined to correspond to facial characters 
(converts emoticons to sound, col. 3, lines 65-67); 

a speech synthesizer for outputting synthesized speech based on the analysis 
results of the text analyzer (text-to-audio transcoding, col. 3, lines 60-61 ), wherein the 
facial character reading assignment unit is constituted by; 

determines if text is a command using an outline symbol table (commands 
placed in brackets, col. 4, lines 16-18); 

wherein the facial character reading assignment unit is constituted by a facial 
character determining unit for determining whether or not a symbol is a symbol 
constituting a facial character (converts emoticons to sound, therefore necessarily 
detecting them, col. 3, lines 65-67); and 

a characteristic extraction unit for extracting characteristic symbols used in facial 
characters using a characteristic symbol table from character strings determined to be 
facial characters (multiple emoticons are recognized and these would necessarily be in 
a symbol table for searching, col. 3, lines 47-53). 

Dutta does not teach analyzing Japanese text data. 

However, the Examiner takes Official Notice that analyzing Japanese text for 
synthesis is well-known in the art. Therefore, it would have been obvious to one of 



Application/Control Number: 09/964,428 Page 8 

Art Unit: 2655 

ordinary skill in the art at the time of invention to modify the system of Dutta to analyze 
Japanese text data because it would allow the system to be used by those who do not 
speak English. 

Dutta does not teach using an outline symbol table to determine whether a 
symbol is a symbol constituting a facial character. 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the system of Dutta to determine whether a symbol is a symbol 
constituting a facial character using the outline symbols for the commands because this 
would allow the system to quickly and easily differentiate between facial characters and 
text so that these facial characters are not processed as regular text. 

Dutta does not teach that the characteristic symbol table consists of 
characteristic symbols and number of groups the characteristic symbols belong to. 

However, the Examiner takes Official Notice that storing data with pointers to 
corresponding data is common in the art. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time of invention to modify the system of Dutta so that 
the characteristic symbol table consists of characteristic symbols and number of groups 
the characteristic symbols belong to because it would save memory. 

Dutta does not teach a reading selection unit for selecting and outputting 
readings for typical vectors most similar to the extracted frequency vectors and that the 
characteristic extraction unit comprises a frequency vector calculator for calculating 
frequencies of characteristic symbols within the facial characters and making frequency 
vectors and a normalization processor for normalizing the frequency vectors. 
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Challapali suggests the facial characters would have different punctuation counts 
(emoticons would have different punctuation counts because the characters that 
comprise them differ, Appendix). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of invention to modify the system of Dutta to have a reading selection unit for 
selecting and outputting readings for typical vectors most similar to the extracted 
frequency vectors and that the characteristic extraction unit comprises a frequency 
vector calculator for calculating frequencies of characteristic symbols within the facial 
characters and making frequency vectors and a normalization processor for normalizing 
the frequency vectors because it would be a robust alternative to matching character 
strings. 

9. As per claim 4, Dutta teaches the facial character reading assignment unit 
decides upon readings for facial characters using the steps of: 

scanning the text and detecting a left outline symbol and right outline symbol 
table (detect left and right brackets to determine command, col. 4, lines 16-18); and 

extracting characteristic symbols used in facial characters from character strings 
encompassed by the left outline symbol and the right outline symbol (converts 
emoticons to sound, therefore necessarily detecting them, col. 3, lines 65-67). 

Dutta does not teach extracting and normalizing frequency vectors indicating 
numbers of appearances of the characteristic symbols, selecting typical vectors most 
similar to the normalized frequency vectors and taking readings allotted to the typical 
vectors as facial character readings. 
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Challapali suggests the facial characters would have different punctuation counts 
(emoticons would have different punctuation counts because the characters that 
comprise them differ, Appendix). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of invention to modify the system of Dutta for extracting and normalizing frequency 
vectors indicating numbers of appearances of the characteristic symbols, selecting 
typical vectors most similar to the normalized frequency vectors and taking readings 
allotted to the typical vectors as facial character readings because it would be a robust 
alternative to matching character strings. 

10. As per claim 5, Dutta teaches a text to speech synthesizer, comprising: 

a facial character reading assignment unit for assigning sounds to character 
string portions of text analysis results determined to correspond to facial characters 
(converts emoticons to sound, col. 3, lines 65-67); 

a speech synthesizer for outputting synthesized speech based on the analysis 
results of the text analyzer (text-to-audio transcoding, col. 3, lines 60-61), wherein the 
facial character reading assignment unit is constituted by; 

determines if text is a command using an outline symbol table (commands 
placed in brackets, col. 4, lines 16-18); 

wherein the facial character reading assignment unit is constituted by a facial 
character determining unit for determining whether or not a symbol is a symbol 
constituting a facial character (converts emoticons to sound, therefore necessarily 
detecting them, col. 3, lines 65-67); and 
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a characteristic extraction unit for extracting characteristic symbols used in facial 
characters using a characteristic symbol table from character strings determined to be 
facial characters (multiple emoticons are recognized and these would necessarily be in 
a symbol table for searching, col. 3, lines 47-53). 

Dutta does not teach analyzing Japanese text data. 

However, the Examiner takes Official Notice that analyzing Japanese text for 
synthesis is well-known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time of invention to modify the system of Dutta to analyze 
Japanese text data because it would allow the system to be used by those who do not 
speak English. 

Dutta does not teach using an outline symbol table to determine whether a 
symbol is a symbol constituting a facial character. 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to modify the system of Dutta to determine whether a symbol is a symbol 
constituting a facial character using the outline symbols for the commands because this 
would allow the system to quickly and easily differentiate between facial characters and 
text so that these facial characters are not processed as regular text. 

Dutta does not teach that the characteristic symbol table is lined up based on 
similarities between shape characteristics. 

Challapali suggests the facial character table is lined up based on similarities 
between shape characteristics (emoticons lined up in order of appearance, Appendix). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of invention to modify the system of Dutta so that the characteristic symbol table is 
lined up based on similarities between shape characteristics because it would limit 
searching for similar characteristic symbols hence improving system speed. 

Dutta does not teach the characteristic extraction unit comprises a frequency 
vector calculator for calculating the frequency of characteristic symbols within the facial 
characters and extracting frequency vectors and a normalization processor for 
normalizing frequency vectors. 

Challapali suggests the facial characters would have different punctuation counts 
(emoticons would have different punctuation counts because the characters that 
comprise them differ, Appendix). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of invention to modify the system of Dutta to have characteristic extraction unit 
comprises a frequency vector calculator for calculating the frequency of characteristic 
symbols within the facial characters and extracting frequency vectors and a 
normalization processor for normalizing frequency vectors because it would be a robust 
alternative to matching character strings. 
11. As per claim 6, Dutta teaches: 

scanning the text and detecting a left outline symbol and right outline symbol 
(detect left and right brackets to determine command, col. 4, lines 16-18); and 
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extracting characteristic symbols used in facial characters from character strings 
encompassed by the left outline symbol and the right outline symbol (converts 
emoticons to sound, therefore necessarily detecting them, col. 3, lines 65-67). 

Dutta does not teach extracting the frequency vectors indicating numbers of 
appearances of the characteristic symbols and normalizing the frequency vectors after 
filtering processing, selecting typical vectors most similar to the normalized frequency 
vectors, and taking readings allotted to the typical vectors as facial character readings. 

Challapali suggests the facial characters would have different punctuation counts 
(emoticons would have different punctuation counts because the characters that 
comprise them differ, Appendix). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of invention to modify the system of Dutta for extracting the frequency vectors 
indicating numbers of appearances of the characteristic symbols and normalizing the 
frequency vectors after filtering processing, selecting typical vectors most similar to the 
normalized frequency vectors, and taking readings allotted to the typical vectors as 
facial character readings because it would be a robust alternative to matching character 
strings. 

Conclusion 

12. The prior art made of record and not relied upon is considered pertinent to 
applicants disclosure. Chin et al. (U.S. Pat. Pub. 2001/0029455A1), Pentheroudakis et 
al. (U.S. Pat. Pub. 2003/0023425A1), Kurlander et al. ("Comic Chat"), Lavine et al. (U.S. 
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Pat. Pub. 2001/0049596A1), Powell (U.S. Pat. 6,157,905), Richardson et al. (U.S. Pat. 
5,812,126), Rosenblatt et al. (U.S. Pat. Pub. 2002/0007276), and Sun (U.S. Pat. 
5,802,482) teach speech processing systems that extract facial characters from text. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Matthew J Sked whose telephone number is (703) 305- 
8663. The examiner can normally be reached on Mon-Fri (8:00 am - 4:30 pm). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Talivaldis Smits can be reached on (703) 306-301 1 . The fax phone number 
for the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 



Business Center (EBC) at 866-217-9197 (toll-free). 
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